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Abstract 

This quasi-experimental research study evaluated two intact undergraduate engineering statics classes at a private 
university in Texas. Students in the control group received traditional lecture, readings and homework assignments. 
Those in the experimental group also were given access to a complete set of online video lectures and videos 
demonstrating how each type of mathematical problem was solved through the learning management system. Overall, 
students in the experimental group perceived course material more positively than those in the control group and felt 
the additional material was useful. The experimental group had slightly improved cumulative test scores, but these 
differences were not statistically significant. 
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1. Introduction 

This study compares the effectiveness of using online supplemental material on a companion Blackboard site to 
enhance the face-to-face classroom environment in an undergraduate engineering Statics class. A total of two 
undergraduate sections of an engineering statics class was studied during the Fall, 2013 semester; these intact classes 
were taught by the same professor. The control group received the traditional lecture in the face-to-face classroom. The 
experimental group had supplemental video material available for each week. Both groups took the same final exam 
and completed survey instruments. 

1.1 Literature Review 

Students, particularly in STEM fields, are tasked with learning an inordinately large amount of material in a short 
amount of time. This can result in only lower levels of Bloom’s taxonomy being obtained (knowledge and 
comprehension) rather than higher levels of understanding (application, evaluation and creation/synthesis) (Hamdan, 
McKnight, McKnight, & Arfstrom, 2013). One way to move from teacher-centered learning to student-centered 
learning where higher taxonomies of learning can occur, is to offer students online supplementary materials they can 
access before class and for test preparation. Implementing new methods and technologies in the classroom may be no 
easy task according to Lawrence & Lentle-Keenan (2013). They note three competing tensions: technology, the teacher 
and the institution greatly impact the success using technology in the classroom, or teaching in a different manner. An 
additional factor noted by Herreid and Schiller (2013) is the students themselves may be resistant to learning in a new 
way. 

Ruthven (2012) noted the efficacy of using additional online materials in technically difficult courses such as 
mathematics. Students are able to access the material any time of the day or night. In addition, class lectures that were 
previously delivered once in the classroom can be viewed over and over again by students to assimilate the material. 
Day et al. (2012) found multimedia education can be effective in science as supplementary resources. According to 
Jordan, Pakzad, and Oats (2010) students in engineering should be “encouraged to expand their knowledge of the 
material being taught through media, images, animation and streaming audio/video” (p. 1). They note these materials 
have the ability to reach more students successfully. They conducted an in-depth survey of 20 engineering students and 
5 faculty. Overall, respondents felt those who attend in a traditional class setting do better than those who attend class 
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online exclusively in engineering education. However, these respondents also noted the benefit and flexibility of 
viewing lectures from the comfort of their own homes. 

Lemley and Jassemnejad (2012) studied the use of Pencasts as supplementary online lecture material in an engineering 
heat transfer course. The Pencast captures written notes on grid paper which can be videoed with audio capture; this 
allowed students to work through difficult problems and understand each step. The group of 13 students gained 15% on 
their Heat Transfer Concept Inventory (HTCI) scores. 

Kao (2008) used an asynchronous online tool to create supplementary material and record classroom lectures and 
converted them into podcasts. He supplied short 5-20 minute supplementary online materials for each week in 
conjunction with the usual face-to-face lectures. He used technology that captured audio, video, PowerPoints and 
handwritten text to create the material; 90% of a population of 20 students found this technology useful. Green, 
Pinder-Grover and Millunchick (2012) commented that video is perceived by students to be beneficial and results in 
increased course performance. They surveyed a population of 397 engineering students in regards to using 5-10 minute 
video screen castings. Of the students surveyed, 90% found homework problem videos to be very helpful and 89% 
used them for exam preparation. Owston, Lupshenuyk and Wideman (2011) noted lower-achieving students tend to 
watch supplementary video material multiple times. In Green et al.’s (2012) study, students that viewed screencasts 
from start to finish had the lowest homework grades, although this was not significant. All students benefitted 
regardless of race, gender and citizenship. In previous research done by the same authors (Pinder-Grover, Green & 
Millunchick, 2011), the number of web hits on such videos correlated to higher test performance. 

Lim, Low, Atallah, Cheang, and LaBoone (2012) discussed a university in Singapore with blended engineering 
programs that include lecture delivery via WebEx technology. The majority of physical class time was devoted to 
laboratories and practical exercises. This program has successfully graduated 513 students in this program since 2008. 
At Curtin University in Perth Australia, Dong, Lucey and Leadbeater (2012) discussed the use of Elluminate Live with 
a visual aid graphics tablet to conduct review sessions for students on the Internet. This supplementary online 
opportunity for students in engineering mechanics was thought to be very beneficial by students. 

A group from the Columbia University School of Continuing Education conducted a qualitative research project where 
they recruited 10 students to engage in an in-depth, targeted interview about video use in education. Data analytics 
from the learning management system and Kaltura were also used to determine video usage by the students. This group 
found videos with a high number of views have a direct correlation to course assessments. In other words, students are 
likely to view videos that show them how to do something, such as mathematical calculations. This group also 
recommends brevity in video presentation, as well as imagery. The average student viewing time was four minutes and 
92% of students viewed the videos on their computer with only 8% using tablets or mobile devices. Students in this 
study appreciated instructor humor and enjoyed videos where the instructor talked them through the mathematical 
calculations. Video production quality was not found to be as important as an engaging, interesting instructor. 
Conclusions from this study were that videos should tie directly to course assignment, faculty members should be 
interesting and engaging, videos should not convey information students can read in the text and four-minutes is the 
optimal video length (Hibbert, 2014). Guo (2013) notes the videos should not exceed 5-10 minutes. 

1.2 “Flipping the Classroom” 

Because of the difficulty of providing quality experiences in the online format, development of online courses in 
STEM fields has still been somewhat limited at this time. However, blended formats are becoming much more popular. 
In the “flipped learning” model, students have the opportunity to review all of the online material in a course before 
coming into the classroom for lecture (Berrett, 2012; Brame, 2014; Herreid & Schiller, 2013; Hamdan et al., 2013; 
Leask, 2014; Shumski, 2014; Stanley & Lynch-Caris, 2014; Toto & Nguyen, 2009; Warter-Perez & Dong, 2012). In 
addition, the student has continual access to these materials throughout the course. In this model, the responsibility for 
the basic course material rests with students (Restad, 2013). Class time is used to do interactive activities and explain 
difficult concepts. While lecture in the classroom is a passive process, classroom work should be an active process in 
the classroom where robust material is provided prior to class to the professor can guide the students through the 
evaluation and solving of complex issues. Hamdan et al. (2013) found in a survey of 454 teachers that 67% of them 
believed the flipped classroom design increased test scores. However, studies that research the true efficacy of this 
flipped classroom are solely lacking in the literature as noted by Goodwin and Miller (2013). 

2. Method 

This research study conducted in Fall 2013 used a quasi-experimental research model with one experimental group that 
received online supplemental material for each module in a Statics class, in addition to face-to-face lecture, and a 
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control group who were exposed to the face-to-face lecture only. Two intact classes were used in this study; 
participants were randomly assigned based on the class they choose to enroll in. The control group contained 31 
students (26 males and 5 females). The experimental group contained 33 students (26 males and 7 females). Students in 
the control group were asked not to obtain the supplementary material from students in the experimental group since a 
research study was being conducted; however, the material was made available to them after the completion of the 
study. The supplemental videos were password protected to prevent data contamination between the two groups. The 
students were solicited to participate in this project by their professor and informed consent was obtained. Participants 
in either group could ask to be excluded at any time. 

The instruments were all created by the researchers based on the research questions and current literature in the field. A 
panel of experts reviewed the instalments to increase content validity. The main purpose of the study was to see if there 
was any difference in grades between the control and experimental group as well as to determine if the use of 
supplementary videos increased or decreased students’ positive perceptions about the class and its content. The 
research questions were as follows: 

1) Would additional supplemental material in the online class increase student’s overall grade point average (GPA) as 
measured in two cluster samples composed of an experimental and control group? 

2) Would student perceptions about the course differ between those who had access to supplementary online material 
and those that did not? 

3) Were there any significant differences in any single student perception in the control and experimental group 
regarding number of course materials, class preparation, class motivation and class difficulty? 

In addition to evaluating the final exam scores of the control and experimental groups, students in the experimental 
group were also asked to participate in an opinion survey about the online supplemental material. There were eight 
Likert-type questions with the following scale: strongly agree, agree, neutral, disagree and strongly disagree and five 
demographic questions. The Cronbach alpha statistic, which is a measure of instalment validity, on this survey was .61 
which is somewhat low; but, the instalment was exploratory in nature so this was expected. With question 5 removed 
(“the course was motivating,’’ the Cronbach alpha score increased to 0.76. Students in both groups were administered a 
second 12- question survey that asked about the course and their study habits. The Cronbach alpha statistic on the 
second instalment assessing the videos was 0.83 which was acceptable for an exploratory instalment. The instalments 
are included in Appendix A. 

Descriptive statistics were obtained from the two surveys and demographic information was collated. The means of 
relevant questions from the instalment assessing student perceptions of the class were compared using a t test. A / test 
was also performed to compare the final exam scores, cumulative test score and individual major test scores from the 
control and experimental groups. 

3. Results 

The control group was 78% male, 12% female and 11% did not specify gender. The mean age was 20.22 years. 
Members of this group were asked to participate in the initial survey given to both the control and experimental groups. 
Eleven participated for a return rate of 28%. The self-reported grade point average (GPA) mean was 3.67; the 
self-reported college freshman GPA was 3.65 and the self-reported engineering GPA was 3.69 out of 4.0. Key survey 


responses can be found in Table 1.0 below: 

Table 1. Control Group Survey Means (1 = Strongly Disagree; 5 = Strongly Agree) 

Question Mean 

1.1 reviewed all assignment material before coming to class. 3.11 

2. Too many course materials were provided. 2.56 

3.1 had the information I needed to be prepared prior to coming to each class. 3.78 

4. The course materials provided assisted me in better learning the concepts of the class. 3.33 

5. This course was motivating. 3.67 

6. This course encouraged me to learn. 3.78 

7. The class went by so fast I had a hard time keeping up. 2.11 
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There were 33 students in the experimental group; 78% were male and 22% were female. The self-reported high school 
GPAofthe five survey respondents was 3.79 out of 4.0. The freshman year of college GPA self-reported mean was 3.71; 
the self-reported mean GPA in engineering courses was 4.00. The mean score for the each survey questions is displayed 
below. The experimental group had access to the regular course material, as well as online lecture video and solution 


videos that showed the students how to work various types of problems step-by-step. 

Table 2. Experimental Group Survey Means (1 = Strongly Disagree; 5 = Strongly Agree) 

Question Mean 

1.1 reviewed all assignment material before coming to class. 2.40 

2. Too many course materials were provided. 1.80 

3.1 had the information I needed to be prepared prior to coming to each class. 4.00 

4. The course materials provided assisted me in better learning the concepts of the class. 4.60 

5. This course was motivating. 4.20 

6. This course encouraged me to learn. 4.40 

7. The class went by so fast I had a hard time keeping up. 1.80 


The students in the experimental group were less likely to review material before coming to class (M=2.40, SD= 0.89) 
than those in the control group (M= 3.11, SD=1.36). A t test was also performed to compare the means of question #2 
“Too many course materials were provided” in the control and experimental groups. The t test was not statistically 
significant (t= 1.14; p =0.28). 

Finally, a t test was also done to compare the means of question #3 “I had the information I needed to be prepared prior 
to coming to each class” between the two groups. The t test value was (t= -0.29, p=Q.ll) which was not statistically 
significant. 

The means for question #4 “The course materials provided assisted me in better learning the concepts of the class” in 
the control and experimental group were compared using a t test to look for statistically significant differences. This 
responses to this question was not statistically significant (t= -1.89, /?=0.08) either. However, the mean of the 
experimental group was considerably more positive (M= 4.60) than the mean of the control group (M= 3.33) 

Overall, the control group answered more negatively on this survey than the experimental group. Students slightly 
disagreed in both groups there were too many course materials provided; the mean was slightly higher in the 
experimental group (M=1.80) than in the control group (M = 2.56). 

Students in the experimental group were also given an additional survey to assess their perceptions of the supplemental 
video lectures that were made available to them. The means for these questions are displayed in Table 3 below. 

Table 3. Experimental Group Assessment of Online Supplemental Videos (l=Strongly Disagree; 3 Neutral, 5 = 


Strongly Agree) 

Question Mean 

1. The video examples greatly assisted me in learning the course material. 2.75 

2. An appropriate number of videos were included in the course. 3.50 

3.1 frequently replayed segments of the videos to study the parts I was having difficulty with. 3.75 

4. The videos were professionally made. 3.75 

5. The videos were too long. 3.25 

6.1 liked the flexibility of having online resources. 3.50 


The students were somewhat positive to neutral in their assessment of the videos in general. They agreed the number 
was appropriate and they were professionally made. Students noted they frequently replayed segments when they were 
having difficulty understanding the material. 

In addition, the scores on three course tests and the final exam for the control and experimental groups were compared 
using a t test. One outlier case was removed from the experimental group since this student has a score of less than 20 
overall and did not participate in class. Scores of the control and experimental group on Test 1 yielded (t = -0.90, p = 
0.37); this was repeated for Test 2 (/ = 0.02, p = 0.99), Test 3 (t = -0.36, p = 0.72) and the Final Exam (t = 0.34, p = 
0.74). In addition a t test was done to compare the overall mean scores on all tests in the control and experimental 
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groups (t = -0.31, p = 0.76). None of the comparisons were statistically significant so the scores students in the 
experimental group who had free access to supplementary online material to prepare and study for this course did not 
differ appreciably from the students in the control group who did not have access to these materials. However the 
experimental group’s mean score on all four tests combined was higher (M = 80.05 ) than the overall mean score for all 
four tests for the control group (M = 79.06). Please see Table 4. 

Table 4. Mean Comparison of Student Test Scores: Control and Experimental Groups 


Test 

M Control Group 

M Experimental Group 

1 

75.77 

79.72 

2 

84.65 

84.59 

3 

78.65 

80.06 

Final Exam 

77.19 

75.84 

Cumulative Test Score 

79.06 

80.05 


The professor who was teaching these two classes was unable to teach the material for one lecture on “3D Moments.” 
An alternate professor was used for the control group class. The experimental group was assigned the videos only and 
never had a face-to-face lecture on this topic. There were two questions on the final exam on this topic. The means of 
the control and experimental group on these two test questions is displayed below in Table 5. The means for these two 
test questions were essentially equal whether the material was delivered by a faculty member in the control group or 
through supplemental videos with the experimental group. 

Table 5. Mean Comparison on Lesson Delivered by Video Only in Experimental Group 


Problem 

M Control Group 

M Experimental Group 

1 (5 points possible) 

3.68 

3.56 

2 (20 points possible) 

14.39 

14.75 


Table 6 below displays the average experimental group viewing times in hours and minutes for the required and 
optional lessons. Students did not watch all of the videos; they only accessed the material they needed. 


Table 6. Experimental Group Video Viewing Times 


Video Topics 

Total Length of 
Videos 

Number of Views/Estimated 
Time Watched 

Time Watched per Student 
in Experimental Group 

Required (Topic 18) 

54 min. 

112/11 hrs. 29 min. 

20 min. 

Not Required (Topic 1-17) 

15 hrs. 35 min. 

391/31 hrs. 5 min 

55 min. 

Total 

16 hrs. 29 min. 

503/42 hrs. 34 min. 

75 min. 


4. Discussion 


In this research study, the use of supplementary online materials did not yield any statistically significant differences 
between the experimental and control group course performance as measured by three tests and a comprehensive final 
exam. The findings of this research study did support the finding that an increase in test scores occurs in a statics course 
with the use of supplementary material as noted by Lemley and Jassemnejad (2012) in a heat transfer course and 
Pinder-Grover et al. (2011) in an engineering study; however, this increase was not statistically significant. 

As noted by Lawrence and Lentle-Keenan (2013), developing supplemental content was not an easy task in this statics 
class. It required extensive work to create multiple lecture and sample problem voiceover videos. Statics is a difficult 
and time-consuming course that must be taken by all engineering students at the university where the research took 
place. Most engineering students at this university carry a 15-18 credit hour load, yet those who responded in the 
experimental group noted they did use the supplementary material. The students in the experimental group reported 
they were more prepared than those in the control group even though they prepared in advance less for class. This 
supported the findings of several researchers (Day et al., 2012; Dong et al., 2012; Green et al., 2012; Jordan et al., 2010, 
Kao, 2008; Ruthven, 2012) who note videos and supplemental information can be beneficial. The students also agreed 
an appropriate number of videos were provided. However, students may be more resistant to learning in a new way as 
noted by Herreid & Schiller (2013) which may have attributed to the lack of statistical difference in test scores in this 
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population. 

There were limitations in this study that impacts its applicability to other populations. First, two intact classes were 
used as a comparison at one university. Because this type of sample was used, the groups were not necessarily 
equivalent and may have differed somewhat because students self-selected into their specific classes. Because of 
variances and preference in student class times, students who did not plan ahead may have ended up in the control 
group. In addition, self-report instruments were used which in and of themselves are limitations. In addition, the 
number of students who responded to the survey was small which can cause an inherent bias, particularly if the 
students who completed the survey were more positive or negative towards the intervention. Truly, it is those students 
in the middle we seek to find out about, but often do not because of limited participation in this type of research. 

Although the study findings were not statistically significant, it provided a frame of reference for improvements in this 
course. One improvement that needs to be made is to shorten the length of the lecture video and deliver this information 
in short clips no more than 10 minutes long (Guo, 2013; Hibbert, 2014). This will increase the number of videos, but 
will allow students to view the sections they want without searching through a long video. This study also supported 
the findings of the Columbia University School of Continuing Education (Hibbert, 2014), since students were more 
likely to access the videos where the instructor solved mathematical calculations rather than lecture videos. 

More research is needed in the analysis of video, both in online learning, and in its use as supplemental material for 
face-to-face classrooms. This is particularly true for highly technical fields such as engineering. However, caution 
must also be taken so students are not inundated with so many additional resources that they cannot access all of them 
and feel overwhelmed. More data is needed on the targeted use of video, including student use and viewing times. This 
is needed to ensure faculty do not spend a great deal of time creating things students will never use. Last, a larger scale 
quantitative study on how students view the use of video in online, blended, flipped and traditional classroom would 
shed light on how effective video truly is for students. 

5. Conclusion 

Overall, the use of supplementary materials in a statics course had no effect on test scores; test scores remained 
equitable. The overall mean test score for the experimental group was slightly higher than the control group but this 
was not significant. The students in the experimental group were more likely to feel they had the materials they needed 
than those in the control group. When students in the experimental group were required to use the videos when lecture 
was for this group had to be cancelled and another faculty member was used to teach the control group, those in the 
experimental group were just as successful as those who received the lecture material in class as measured by two exam 
problems. In this study, the use of supplementary online material was found to be useful to the students who had access 
to it. It can be invaluable when lecture must be cancelled because a professor is not available. If this supplementary 
material is available, valuable class time need not be lost due to instructor illness. In addition, this material can also be 
provided to students who had to miss class due to illness or other obligations to ensure they have access to all needed 
materials. 
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APPENDIX A 
SURVEY INSTRUMENTS 


ENGINEERING ONLINE STATICS CLASS UNDERGRADUATE STUDY 
STUDENT QUESTIONAIRE #1 

(Administered to All Students in Control and Experimental Groups) 

Please answer all questions using the following rating scale: Strongly agree, Agree, Neutral, Disagree, Strongly 
Disagree 


■ I reviewed all assigned material before coming to class.(Restad, 2013) 

■ Too many course materials were provided. (Goodwin & Miller, 2013) (reverse code) 

■ I had the information 1 needed to be prepared prior to coming each class. (Restad 2013) 

■ The course materials provided assisted me in better learning the concepts of this class. (Restad, 2013). 

■ This course was motivating. (Miller, 2012). 

■ This course encouraged me to learn. (Miller, 2012) 

■ Too many course materials were provided (Goodwin & Miller, 2013). (reverse code) 

■ The class went so fast 1 had a hard time keeping up (Goodwin & Miller, 2013) (reverse code) 


Demographic Information 

■ What is your age?_ 

■ What is your gender_ 

■ What is your self-reported high school GPA on a scale of 0-4.0 (numerical such as 3.5)? 

■ What was your college freshman GPA?_ 

■ What was your GPA in your freshman engineering classes ONLY_ 
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ENGINEERING ONLINE STATICS 
STUDENT FEEDBACK QUESTIONAIRE #2 
(Administered only to Experimental Group) 

Please answer all questions using the following rating scale: Strongly agree, Agree, Neutral, Disagree, Strongly 
Disagree 

Video Content 

Lecture videos were used to introduce new content, while example videos were used to demonstrate the use of the 
concepts in a practice problem. 

7. The video lectures greatly assisted me in learning the course material. 

8. The video examples greatly assisted me in learning the course material. 

9. I frequently took notes as the videos played. 

10. I frequently worked ahead of the example videos before watching the solution. 

11. An appropriate number of videos were included in the course. 

12. I found the videos boring and an ineffective way to learn the material. 

13. I frequently replayed segments of videos to study the parts I was having difficulty with. 

14. The videos were professionally made. 

15. The videos were too long. 

Other Content/Format 

16. The homework problems greatly assisted me in learning the course material. 

17. The assessments greatly assisted me in learning the course material. 

18. 1 liked the flexibility of this course having online resources. 
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